Image processing apparatus, image processing method, program, and recording medium

ABSTRACT

An image processing apparatus capable of automatically adjusting the enlargement ratio and the position of images and to display the noticed areas of all selected images is disclosed. When a two image display mode is selected, an image previously displayed (a first image) and an image to be displayed in the two image display mode (a second image) are loaded from a memory into a display controlling unit and a correction value calculating unit. The display controlling unit performs enlarging and moving operations with respect to the first image and the second image simultaneously. The correction value calculating unit searches for an area in the second image (a corrected area) most resembling an enlarged area in the first image so as to enlarge the corrected area in the second image. The images processed by the display controlling unit are transmitted to an image display memory.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to a digital image processingapparatus, and more specifically to a technique preferably adapted tosoftware for a printer, a facsimile machine, a digital camera, apersonal computer, and the like.

2. Description of the Related Art

Recently, the image resolution of image input devices such as a flat bedscanner and a digital camera has been remarkably improved. However, theimage resolution of display devices displaying the images input by suchinput devices is not so high as that of the input devices. For example,there are so many digital cameras having resolutions of 4 through 5million pixels, but the resolutions of liquid crystal displays forpersonal computers are still 1.3 million pixels for SXGA and 2 millionpixels for UXGA. Further, the resolution of a liquid crystal displayattached to a digital camera is even lower. As a result, the resolutionof an image input by an image input device needs to be reduced whenviewed on a display.

Unfortunately, when the image is viewed with reduced resolution,necessary information may not be obtained in many cases. For example, acase where a picture of a person is taken with a digital camera isconsidered. When pictures are taken, plural same pictures may be takento get a best shot. However, in typical use of such a digital camera,since the data storage capacity of the camera is limited, the user ismore likely to delete some unnecessary plural pictures when the capacityis running out. In this case, the user may view the attached liquidcrystal display to select pictures to be deleted. Unfortunately, theremay be many cases where the eyes of the person are closed when thepictures are viewed on a display of a personal computer. To avoid suchproblems, there may be a method of enlarging a part of a picture thathas been shrunk. However, there is a problem that it is very troublesometo perform the same operations of enlarging and moving with respect toplural pictures.

To solve the problem, Patent Document 1 discloses a technique in whichselected plural images are displayed on the same display; when enlarge,shrink, and move operations are performed on one of the plural pictures,the same operations are also applied to the other pictures displayed onthe display. By doing this, it is possible to determine whether the eyesof the person are closed with fewer operations. In addition, it is notto hard to anticipate that this technique of Patent Document 1 is usefulwhen applied to a document image scanned by a flat bed scanner. Forexample, when this technique is applied to document images viewed as thesame document when viewed in a list in a thumbnail grid mode, it ispossible to effectively view a selected part of each document whereimportant information is included.

Patent Document 1: Japanese Patent Application Publication No. 2006-5640

Non Patent Document 1: 1C. Cortes and V. N. Vapnik “Support VectorNetworks,” Machine Learning, vol. 20, pp. 273-297, 1995

Non Patent Document 2: N. Otsu, and T. Kurita, “A new scheme forpractical, flexible and intelligent vision system,” Proc. IAPR Workshopon Computer Vision, pp. 431-435, 1988.

However, it is not necessarily the case that all the document imagesscanned by a flat bed scanner have exactly the same layout.

As an example, FIGS. 1A through 1D illustrate such a case where asnapshot (picture) of a person is taken. FIG. 1A is a first picture ofthe person. As shown in FIG. 1A, the person is displayed in the centerof the picture. FIG. 1B is an enlarged image of the area surrounded bythe rectangle in FIG. 1A. After a few seconds, a next picture is takento get a best shot. FIG. 1C is a second picture of the person. As shownin FIG. 1C, the person is moved to the left-hand side of the picture.Now, FIG. 1D is another enlarged image of the area surrounded by therectangle in FIG. 1C. The positions of the rectangles in FIGS. 1A and 1Care the same with respect to the corresponding pictures. As a result, asshown in FIG. 1D, the face of the person is not included in FIG. 1D. Insuch a case like this, it is necessary to slightly adjust the enlarged(noticed) area so as to see the face of the person as shown in FIG. 1B.As described, there may be cases where necessary information is notincluded in the same noticed areas when the position is slightly movedamong plural pictures.

The above case may be relatively extreme because the person is moved alarge distance from the center position, and the above case may not belikely to occur in a typical photo shoot. However, when an enlargementfactor is large, that is, when the noticed area is relatively smallcompared with the original image including the noticed area, the sameproblem as described above may occur. For example, a display deviceattached to a typical digital camera is very small. Therefore, to checkan image on the display device carefully, the image is required to besufficiently enlarged. As another example, there may be a case where theface of a person is very small when, for example, a group of people areincluded in a picture. In such a case, it is also required tosufficiently enlarge the necessary area of the picture. When these casesare considered, a camera shaking in a photo shoot may become a seriousmistake in the photo shoot.

SUMMARY OF THE INVENTION

The present invention is made in light of the above problems, and mayprovide an image processing apparatus, an image processing method, aprogram, and a recording medium capable of simultaneously displayingpartially enlarged plural images on a display, and for an operation suchas changing the enlargement factor and moving the noticed area in one ofthe plural images, automatically performing the same operation performedin the one of the plural images, displaying the noticed areas of the allimages, and improving the usability.

According to an aspect of the present invention, there is provided animage processing apparatus including an image input unit inputting animage; a recording unit recording an image input by the input imageunit; a display unit displaying an image signal recorded in therecording unit; a first display controlling unit simultaneouslyperforming a process selected from the group including enlarging animage and moving an image with respect to specific plural imagesdisplayed on the display unit; and a correction value calculating unitcalculating a correction value with respect to a process in the firstdisplay controlling unit in accordance with the selected process;wherein a process in the first display controlling unit is corrected inaccordance with the correction value.

According to an aspect of the present invention, there are someadvantages as describes below:

(1): In a system where some of partially enlarged images of thecorresponding plural images are simultaneously displayed on a display,and when a user changes an enlarge rate and moves a scope range withrespect to one image, the same operations are performed with respect tothe other images, and the enlarging operation and the moving operationare automatically performed with respect to the other images. As aresult, a user can look at each notice area of all images, therebyimproving the convenience of use.

(2): In a system where some of thumbnail images of plural images and thethumbnail's partially enlarged images are simultaneously displayed on adisplay, and when a user changes an enlarge rate and moves a scope rangewith respect to one image, the same operations are performed withrespect to the other images, the enlarging operation and the movingoperation are automatically performed with respect to the other images.As a result, a user can look at each notice area of all the images,thereby improving the convenience of use.

(3): Since a user can change the enlarge rate and move a scope range, auser can set a notice range as the user likes, thereby improving theconvenience of use.

(4): A user can grasp the entire contents of an image by arranging athumbnail and the thumbnail's partially enlarged image side by side,thereby improving the convenience of use.

(5): For a part of an image having higher probability that a user wouldlike to check it, the user is automatically prompted to check it,thereby reducing the operations of the user and improving theconvenience of use.

(6): Since an image to be simultaneously browsed as a resembled image ispreviously searched for and extracted from images stored in a recordingdevice and is displayed to a user, the convenience of use is improved.

(7) Since a type of an image is determined, classified, and stored in arecording device in advance, efficiency of searching for an image isincreased and the convenience of use is improved.

(8) Since a display mode of browsing a file is switched depending on thetype of the file, the file can be more effectively browsed and theconvenience of use is improved.

(9) Since the type of an image is determined before a similarity levelis determined, the frequency of determining the type of image isreduced, thereby increasing the processing speed.

(10) Since not only position correction information but also resolutioncorrection information are calculated and used, a user doesn't have topay attention to resolution setting when an image is input, therebyimproving the convenience of use.

(11) Since an entirely shrunk image and a partially enlarged image aredisplayed side by side, a user can easily recognize the entire contentsof an image, thereby improving the convenience of use.

(12) Since a user can directly perform operations on a rectangleindicating an enlarged area using a pointing device when an imagepartially enlarged from a part of an image and an image shrunk from anentire image are displayed side by side, the user can intuitivelyperform operations, thereby improving the convenience of use.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, features, and advantages of the present invention willbecome more apparent from the following descriptions when read inconjunction with the accompanying drawings, in which:

FIGS. 1A through 1D illustrate a conventional problem;

FIG. 2 is a drawing showing an exemplary configuration of a digitalcamera according to an embodiment of the present invention;

FIGS. 3A through 3E illustrate a play mode;

FIG. 4 shows an exemplary partial configuration according to a firstembodiment of the present invention;

FIGS. 5A and 5B are drawings showing correction values calculated by acorrection value calculating unit;

FIG. 6 is a flowchart showing a process of simultaneously displayingplural images in a REC view hold mode;

FIG. 7 is a flowchart showing a process of automatically displayingplural images of a image group in a REC view hold mode;

FIG. 8 shows an exemplary configuration according to a second embodimentof the present invention;

FIG. 9 shows an exemplary folder configuration;

FIG. 10 shows a display example displayed according to the secondembodiment of the present invention;

FIG. 11 is a drawing showing an image created by enlarging theleft-upper image in FIG. 10;

FIG. 12 shows patterns of higher-order local autocorrelation;

FIG. 13 illustrates an exemplary configuration of a similar imagesearching unit; and

FIG. 14 is a drawing showing an operation of SVM.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following, exemplary embodiments of the present invention aredescribed with reference to the accompanying drawings.

Embodiment 1

FIG. 2 shows a configuration of an image processing apparatus (a digitalcamera) 100 according an embodiment of the present invention.

As shown in FIG. 2, the image processing apparatus 100 includes ashooting lens 10, a shutter 12, an imaging device 14, an A/D converter16, a timing generator circuit 18, an image processing circuit 20, amemory controlling circuit 22, an image display memory 24, a D/Aconverter 26, an image display section 28, a memory 30, a compressingexpanding circuit 32, an exposure controlling circuit 40, a focuscontrolling circuit 42, a zoom controlling circuit 44, a flash 48, asystem controlling circuit 50, a memory 52, a display section 54, anonvolatile memory 56, a mode dial 60, a shutter switch 63, an imagedisplay ON/OFF switch 66, a REC view ON/OFF switch 68, an operationssection 70, interfaces 90 and 94, a recording medium detecting section98, an optical finder 104, a communicating section 110, a gain amplifier120, and a time measuring section 121.

The shutter 12 has an aperture function. The imaging device 14 convertsan optical image into an electronic signal. The gain amplifier 120amplifies an analog signal output from the imaging device 14 todetermine the sensitivity of the camera. The A/D converter 16 convertsthe analog signal output from the imaging device 14 into a digitalsignal.

The timing generator circuit 18 provides a clock signal and a controlsignal for the imaging device 14, the A/D converter 16, and the D/Aconverter 26. The timing generator circuit 18 is controlled by thememory controlling circuit 22 and the system controlling circuit 50. Theimage processing circuit 20 performs prescribed pixel interpolation andcolor conversion processes with respect to the data from the A/Dconverter 16 or data from the memory controlling circuit 22. Further,the image processing circuit 20 performs a prescribed calculationprocess using imaged image data. Based on the obtained calculationresult, the system controlling circuit 50 controls the exposurecontrolling circuit 40 and the focus controlling circuit 42 to perform aTTL(Through-The-Lens)-type AF(Auto Focus) process, an AE(Auto Exposure)process, and an EF(pre-Exposure Flash) process.

Still further, after performing a prescribed calculation using theimaged image data, the image processing circuit 20 performs a TTL-typeAWB (Auto White Balance) process based on the obtained calculationresult.

The memory controlling circuit 22 controls the A/D converter 16, thetiming generator circuit 18, the image processing circuit 20, the imagedisplay memory 24, the D/A converter 26, the memory 30, and thecompressing expanding circuit 32. Data output from the A/D converter 16are stored in the image display memory 24 or the memory 30 via the imageprocessing circuit 20 and the memory controlling circuit 22, or via thememory controlling circuit 22 only.

The system controlling circuit 50 includes an exposure calculatingsection 122, a luminance level calculating section 123, and a correctiongain calculating section 124. The exposure calculating section 122calculates an appropriate exposure value based on a measured luminancelevel of TTL light via the memory controlling circuit 22, and controlsthe exposure controlling circuit 40. The luminance level calculatingsection 123 calculates a luminance level from the imaged image data viathe memory controlling circuit 22. The correction gain calculatingsection 124 compares the luminance level measured by the exposurecalculating section 122 with the luminance level calculated by theluminance level calculating section 123, and calculates an offset valueso that the luminance level is appropriate. Then, based on the offsetvalue, the image processing circuit 20 produce a digital gain correctionvalue.

The image display section 28 is made of, for example, a TFT LCD. Imagedata to be displayed stored in the image display memory 24 aretransmitted to the image display section 28 via the D/A converter 26,and displayed on the image display section 28. An electronic finderfunction can be realized when the imaged image data are sequentiallydisplayed on the image display section 28.

The display on the image display section 28 can be turned ON/OFF by aninstruction from the system controlling circuit 50. When the display isturned OFF, power consumption of the image processing apparatus 100 canbe greatly reduced. It should be noted that the system controllingsection 50 performs as an image display controlling unit controlling,for example, a display status of the image display section 28. Thememory 30 stores input still images and moving images, having sufficientcapacity for storing a prescribed number of still images and aprescribed time period of moving images. As a result, a large of amountimage data, for example, a sequential shooting pictures in a sequentialshooting mode and a panoramic picture in a panoramic shooting mode, canbe stored in the memory 30 quickly. Further, the memory 30 can also beused as a working area of the system controlling circuit 50.

The compressing expanding circuit 32 compresses and expands image databy, for example an ADCT(Adaptive Discrete Cosine Transform) method. Thecompressing expanding circuit 32 reads image data in the memory 30 andcompresses or expands the read image data, and stores the compressed orexpanded image data in the memory 30. The exposure controlling circuit40 controls the shutter 12 having an aperture function, and can adjustthe amount of light in cooperation with a flash 48.

The focus controlling circuit 42 controls the focus of the shooting lens10. The zoom controlling circuit 44 controls the zoom of the shootinglens 10. The exposure controlling circuit 40 and the focus controllingcircuit 42 are controlled by the system controlling circuit 50 based ona calculation result of the imaged image data from a TTL light by theimage processing circuit 20. The system controlling circuit 50 controlsthe entire image processing apparatus 100. The memory 52 stores, forexample, constant numbers, variable numbers, and programs.

The display section 54 includes a liquid crystal display device and aspeaker providing messages about the operating status using characters,images, and sound in accordance with the execution of a program. One ormore display sections 54 are provided at easily viewed locations nearthe operations section 70. The display section 54 includes a combinationof, for example, an LCD, an LED, and a voice device. Some part of thedisplay section 54 is provided in the optical finder 104.

The display contents displayed on the LCD and the like of the displaysection 54 include a single shot/sequential shooting mode, a self-timer,a compression rate, the number of recording pixels, the number ofrecorded images (pictures), the number of recordable pictures, shutterspeed, aperture value, exposure correction, a flash mode, a red-eyereduction mode, a macro shooting mode, a buzzer setting mode, remainingbattery power for clock, remaining battery power, an error message,information display in plural digits, status whether recording media 200and 210 are attached, operating status of communication I/F, and dateand time.

The display contents displayed on the optical finder 104 among theentire display contents of the display section 54 include focusedstatus, camera shake alarm, flash charged status, shutter speed,aperture value, and exposure correction. The nonvolatile memory 56 iselectrically-erasable and electrically-writable memory such as anEEPROM.

To operate the image processing apparatus 100, the mode dial 60, aswitch SW1 62, the shutter switch 63, a switch SW2 64, the image displayON/OFF switch 66, the REC view ON/OFF switch 68, and the operationssection 70 are used. Instructions to operate various functions are givento the image processing apparatus 100 via the system controlling circuit50. An instruction is given to the image processing apparatus 100 by anysingle operation of a switch, a dial, a touch panel, pointing by visualsight detection, voice recognition, or a combination of those operationssections. More detailed explanations of those operations sections aregiven below.

The shutter switch SW1 62 is turned ON when the shutter switch 63 isbeing operated to give an instruction to start processes such as theAF(Auto Focus) process, the AE(Auto Exposure) process, the AWB(AutoWhite Balance) process, and the EF(pre-Exposure Flash) process. Theshutter switch SW2 64 is turned ON when the operation of the shutterswitch 63 is ended. When the shutter switch SW2 64 is turned ON, aninstruction is given to start a series of processes including anexposing process, a developing process, and a recording process. In theexposing process, a signal from the imaging device 14 is stored as imagedata in the memory 30 via the A/D converter 16 and the memorycontrolling circuit 22. The developing process is performed usingcalculations by the image processing circuit 20 and the memorycontrolling circuit 22. In the recording process, the image data areread from the memory 30 and are compressed by the compressing expandingcircuit 32. Then the compressed image data are stored in the recordingmedium 200 or 210.

The image display ON/OFF switch 66 turns ON/OFF the image displaysection 28. This image display ON/OFF switch 66 allows for cutting offthe current supply to the image display section 28 of, for example, aTFT LCD, thereby reducing power consumption.

The REC view ON/OFF switch 68 controls a REC view (quick review)function automatically playing imaged image data right after theshooting. Especially in this embodiment, it is assumed that the REC viewfunction can be performed even when the image display section 28 isturned OFF. The time measuring section 121 can measure a time intervalbetween when the shutter switch SW1 62 is turned ON and when the shutterswitch SW2 64 is turned ON.

The operations section 70 includes various types of buttons and a touchpanel used as a menu button, a set button, a macro button, a multiimages play and page feed button, a flash set button, a singleshoot/continuous shoot/self-timer mode selection button, a menu shift(−) button, a menu shift (+) button, a playing image shift (+) button, aplaying image shift (−) button, an image quality selecting button, anexposure correction button, and a date/time set button.

The power controlling section 80 includes a battery detecting circuit, aDC-DC converter, and a switching circuit switching a block to be poweredup. The power controlling section 80 detects whether a battery isattached, the type of the battery, and remaining energy of the battery.Based on the detection result and an instruction from the systemcontrolling circuit 50, the power controlling section 80 controls theDC-DC converter to supply necessary voltage to each section includingthe recording medium for a necessary time period.

There is a power section 86 connected to the power controlling section80 via connectors 82 and 84. The power section 86 includes a primarybattery such as an alkaline battery or a lithium battery, a secondarybattery such as a NiCd battery, a NiMH battery, or a Li battery, and anAC adaptor.

The recording medium detecting section 98 determines whether a recordingmedium 200 or 210 such as a memory card or a hard disk is attached tothe image processing apparatus 100 via the interface 90 or 94 connectedto connectors 92 or 96, respectively, providing connection between theinterface and the recording medium.

In this embodiment, it is assumed that the image processing apparatus100 has two interfaces and two connectors each capable of connecting arecording medium. Obviously, the number of such interfaces andconnectors is not limited to two, and the image processing apparatus 100may include one or more such interfaces and connectors. Further, whenplural such interfaces and connectors are provided, the interfaces andconnectors may be based on the same or different standards. For example,the interface and the connector conform to a standard such as PCMCIAcard and CF (Compact Flash (trademark)) card.

Further, when the interfaces 90 and 94 and the connectors 92 and 96conform to a standard such as the PCMCIA card or the CF card, byconnecting a communication card such as a LAN card, a modem card, a USBcard, an IEEE 1394 card, a P 1284 card, a SCSI card, or a communicationcard for PHS, it becomes possible to exchange image data and managementinformation of the image data with other peripheral devices such as acomputer and a printer.

The optical finder 104 allows for shooting using the optical finder 104even without using an electronic finder function of the image displaysection 28. On the optical finder 104, some functions of the displaysection 54 such as focused status, camera shake alarm, flash chargedstatus, shutter speed, aperture value, and exposure correction can bedisplayed.

The communication section 110 has various communicating functions suchas RS 232C, USB, IEEE 1394, P 1284, SCSI, modem, LAN, and wirelesscommunication. Reference numeral 112 denotes a connector when the imageprocessing apparatus 100 is connected to another apparatus via thecommunication section 100 or an antenna when a wireless communicationmethod is used.

The recording medium 200 may be a memory card or a hard disk andincludes a recording section 202 made of, for example, a semiconductormemory or magnetic disks, an interface 204 to the image processingapparatus 100, and a connector 206 to provide connection to the imageprocessing apparatus 100.

The recording medium 210 may be a memory card or a hard disk andincludes a recording section 212 made of, for example, a semiconductormemory or magnetic disks, an interface 214 to the image processingapparatus 100, and a connector 216 to provide connection to the imageprocessing apparatus 100.

In the following, a characteristic feature of a configuration accordingto an embodiment of the present invention in play mode is described indetail. FIG. 3A shows a one image display mode which is the normal playmode. During the one image display mode, when a plural displayinstruction is issued, the display mode goes to a plural display mode.FIG. 3B shows the plural display mode. In the following, the left-handside area in a display mode as shown in FIG. 3B is referred to as “afirst display area”. Similarly, the right-hand side area in the displaymode as shown in FIG. 3B is referred to as “a second display area”. Theplural display mode may be set by pressing or holding down a prescribedbutton on the operations section 70 of the image processing apparatus100, or may be selected from a menu display on the image display section28.

In this description, a case of displaying two images is described forsimplicity purposes. However, it should be noted that an embodiment ofthe present invention is not limited to the case of displaying twoimages as shown in FIG. 3B. Namely, any number of images may bedisplayed. Further, as shown in FIGS. 3B through 3E, on the lower sideof the display there is provided a toggle mechanism indicating whichdisplay is being selected so as to be operated. The display may beselected using the toggle mechanism as shown in FIGS. 3B through 3E.However, a method of selecting the display is not limited to the togglemechanism. For example, a cross button and a set button may be used toselect a display and confirm/cancel the selection of the display,respectively. Otherwise, a touch panel may be used to select a displaywhen an apparatus having a touch panel is used.

In this description, a set button as a toggle switch is used to select adisplay to be operated on. In the display mode of FIG. 3B, when the setbutton is pressed once, the selected display is changed into a one imageselected mode as shown in FIG. 3C. In any mode, any display operation(including enlarging and moving a display) is effective for the selectedwindow(s) only. In the display mode of FIG. 3C, when the set button isfurther pressed once, the selected display is changed into another oneimage selected mode as shown in FIG. 3D. Further, in the display mode ofFIG. 3D, when the set button is pressed once, the selected display ischanged into a two (plural) image selected mode as shown in FIG. 3E. Inthis two (plural) image selected mode, for example, both of the selectedtwo (plural) displays can be enlarged and moved simultaneously. Afterthe displays are enlarged, when the display mode is changed into the oneimage selected mode to display one image, and the other image isselected to be displayed, the selected display has the same enlargedratio and the moved position in accordance with the enlarged and themoved operations that have been made in the two (plural) image selectedmode. Namely, when the next display image is selected, settings of theenlarged ratio and moved position are also applied in the next displayimage.

FIG. 4 is a flowchart showing the above process in detail. When the twoimage selected mode is selected, image data that have been displayedbefore the setting (hereinafter referred to as “a first image”) and theother data that are not displayed before the setting (hereinafterreferred to as “a second image”) are read from the memory 30.

The image data read from the memory 30 are transferred to a displaycontrolling unit 504 and a correction value calculating unit 503.

In the display controlling unit 504, enlarging and moving operations areperformed with respect to both the first and the second images. However,the enlarged area in the first image is not always entirely equal to theenlarged area in the second image. Namely, the enlarged area in thesecond image is a correct area determined based on correction valuescalculated in the correction value calculating unit 503.

Next, the correction value calculated by the correction valuecalculating unit 503 is described. FIGS. 5A and 5B show the first andthe second images side by side. As shown in FIGS. 5A and 5B, originalpoints are set at the bottom left corner of the first and the secondimages, and a coordinate system with the right (horizontal) directionset as an x direction and the up (vertical) direction set as a ydirection is applied to the first and the second images. A user'soperation to enlarge and move an image is equivalent to the userdefining a rectangle area to be enlarged. Namely, when the rectangulararea to be enlarged in the first image is expressed as “R0”, “R0” isgiven as “(xl, yb, xr, yt)”. In the same manner, a rectangular area tobe enlarged in the second image “R1” is given as “(xl+cx, yb+cy, xr+cx,yt+cy)”. The values “cx” and “cy” are correction values calculated bythe correction value calculating unit 503.

In the correction value calculating unit 503, image data included in therectangular area “R0” in the first image are scanned to search for therectangular area “R1” so that the image of the rectangular area “R1”resembles the image of the rectangular area “R0” most closely. From atechnical point of view, the correction values “cx” and “cy” thatmaximize a correlation coefficient “C” are given in the followingformula 1 by scanning the correction values “cx” and “cy”.

$\begin{matrix}{C = \frac{\sum\limits_{x = x_{l}}^{x_{r}}\; {\sum\limits_{y = y_{b}}^{y_{t}}\; {{I_{0}\left( {x,y} \right)}*{I\left( {{x + c_{x}},{y + c_{y}}} \right)}}}}{\sqrt{\begin{matrix}{\sum\limits_{x = x_{l}}^{x_{r}}\; {\sum\limits_{y = y_{b}}^{y_{t}}\; {{I_{0}\left( {x,y} \right)}^{2}*}}} \\{\sum\limits_{x = x_{l}}^{x_{r}}\; {\sum\limits_{y = y_{b}}^{y_{t}}\; {I\left( {{x + c_{x}},{y + c_{y}}} \right)}^{2}}}\end{matrix}}}} & (1)\end{matrix}$

In formula 1, “I₀(x,y)” denotes a pixel value at (x,y) of the firstimage in the coordinate system; and “I(x,y)” denotes a pixel value at(x,y) of the second image in the coordinate system. It should be notedthat the calculation of the correction values is performed only when auser enlarges and moves an image.

The image data processed by the display controlling unit 504 aretransferred to the image display memory 24 to be processed as describedabove and displayed on the image display section 28.

As described above, an area to be enlarged in the second image issearched for so as to maximize the similarity between the image in thearea to be enlarged in the first image and the image in the area to beenlarged in the second image. Therefore, even when positions of anobject image are different between the first image and the second image,the area designated by a user in the first image to be displayed in thefirst display area, which is equivalent to an area that the user needsto check, is extracted from the second image, and the extracted image isdisplayed in the second display area. When this is performed, the userdoesn't need to perform fine adjustment, thereby improving theconvenience of use.

In the above description, correction values are defined only in x and ydirections. However, when the an affine conversion as shown in thefollowing formula 2 is used, corrections in a rotating direction canalso be performed easily.

$\begin{matrix}{\begin{pmatrix}x^{\prime} \\y^{\prime}\end{pmatrix} = {{\begin{pmatrix}{{\cos \; \theta} - {\sin \; \theta}} \\{\sin \; {\theta cos}\; \theta}\end{pmatrix}\begin{pmatrix}x \\y\end{pmatrix}} + \begin{pmatrix}c_{x} \\c_{y}\end{pmatrix}}} & (2)\end{matrix}$

In formula 2, “x” and “y” denote coordinate values in a noticed area;“x′” and “y′” denote coordinate values after the conversion; “θ” denotesa rotation angle; and “cx” and “cy” denote moved distances in x and ydirections, respectively, by the conversion.

Further, in the above description, the correction values are calculatedwhen a first enlarge and move operation is performed. However,correction values may be calculated whenever an enlarge and movingoperation is performed, or whenever a user presses a button assigned tothe function.

Next, a case is considered where the first enlarging and movingoperation is performed in the one image selected mode, then the displaymode is changed to the two (plural) image selected mode, where thesecond enlarging and moving operation is performed. In such a case,before starting the second enlarging and moving operation, the enlargedratio and the positions of an object image among the two (plural) imagesmay be different from each other. In this case, the second enlarging andmoving operation is applied to the two (plural) images that are beingdisplayed just before the second enlarging and moving operation. Forexample, there are two images: one image is enlarged at an enlargementratio of 1 and the other image is enlarged at an enlargement ratio of 2.Then those two images are displayed side by side in the two (plural)image selected mode, and an operation to enlarge at an enlargement ratioof 2 is simultaneously applied to the two images. Then, a two-timesenlarged image and a four-times enlarged image are obtained. However,when the result of an operation would exceed the limit of the enlargingand moving operation, the operation will not be performed. Next, a caseis considered where there are two images; one is a vertically-long imageand the other is a horizontally-long image. When those two images aredisplayed side by side in the two (plural) image mode, and the enlargingoperation is performed on those two images, the enlarged images willalso be a vertically-long image and a horizontally-long image the sameas before the operation. It should be noted that there is no differencein operations between a vertically-long image and a horizontally-longimage. When a next image is to be displayed in accordance with theenlarged ratio and the position of an object image of the currentdisplayed image is unchanged but the next image is a vertically-longimage and the current image is a horizontally-long image or vice versa,the next image will be displayed substantially in accordance with theenlarged ratio and the position of an object image.

The system controlling circuit 50 can exchange the positions of theselected two images by using an operation button on the operationssection 70 or making a selection on a menu. Further, a histogram ofpixels being displayed and a white flight area can be displayed by usingan operation button on the operations section 70 or making a selectionon a menu.

The system controlling circuit 50 can select one or more images beingdisplayed and delete the selected images. In this operation, conversely,it may be arranged that only an unselected one or more images aredeleted. In the same manner, it is possible to select one or more imagesbeing displayed and set protection for the selected images; and it isalso possible to select one or more images being displayed as printtarget images to be directly printed out by an external output deviceconnected to the image processing apparatus (digital camera) 100.

An image displaying method and a displayed image operating method usedin an image play mode of a digital camera according to an embodiment ofthe present invention as described above may also be executed in the two(plural) image selected (displayed) mode on a “REC view hold”. The “RECview” is a function to automatically display shot image data that havejust been shot on the image display section 28 for a certain period oftime. Normally, right after image shooting, the shot image data storedin the image display memory 24 or the memory 30 are read back to bedisplayed on the image display section 28. The “REC view hold” refers toa function to keep on executing the REC view function when a button forholding the REC view on the operations section 70 is pressed during theREC view period, and the REC view function is terminated when a buttonfor terminating the REC view is pressed.

In the description of an embodiment of the present invention, two(plural) resembling images are displayed on the image display section 28and are enlarged and moved simultaneously to be compared to each other.As two (plural) images to be compared to each other displayed during the“REC view hold” mode, bracket images shot in a bracket shooting mode,all the sequential images shot in a sequential shooting mode, or allimages shot in a certain period of time in the normal shooting mode aretypically used. The group of images may be displayed in a manner thatall images are displayed in a two (plural) image display mode or only aprescribed number of images are displayed in each display step.

FIG. 6 is a flowchart showing an operational process of displaying two(plural) images in the “REC view hold” mode.

First, as soon as a shooting starts (step S701), a REC view displaydisplaying the shot image data on the image display section 28 isperformed (step S702). While the REC view display is being performed, itis determined whether the REC view hold is designated (step S703). Whenit is determined that the REC view is not designated, the REC viewdisplay is terminated after a certain period of time (step S707). Whenit is determined that the REC view hold is designated in step S703, theREC view display is continued, and it is determined whether the two(plural) image display mode is designated (step S704). In step S704,when it is determined that the two (plural) image display mode is notdesignated, the process waits until a designation to terminate the RECview hold, and when the REC view hold is terminated, the REC viewdisplay is terminated. When it is determined that the two (plural) imagedisplay mode is designated, two (plural) images are displayed (stepS705). Then, a user performs operations such as enlarging and moving animage as described in the above embodiment to check the images. Afterthe REC view hold is terminated (Yes in step S706), the REC view displayis terminated (step S707).

Further, by previously setting a mode for displaying two (plural) images(multi display mode) during the REC view hold mode, the above group ofimages is automatically displayed during the REC view mode.

FIG. 7 is a flowchart showing an operational process of displaying two(plural) images in the “REC view hold” mode when the multi display modeis previously set. First, as soon as a shooting starts (step S801), aREC view display displaying the shot image data on the image displaysection 28 is performed (step S802). While the REC view display is beingperformed, it is determined whether the REC view hold is designated(step S803). When it is determined that the REC view hold is notdesignated, the REC view display is terminated after a certain period oftime (step S806). When it is determined that the REC view hold isdesignated in step S803, the REC view display is continued and two(plural) images are displayed (step S804). Then, a user performsoperations such as enlarging and moving an image as described in theabove embodiment to check the images. After the REC view hold isterminated (Yes in step S805), the REC view display is terminated (stepS807).

It should be noted that the operations of the above described embodimentof the present invention are performed in a manner so that the systemcontrolling circuit 50 controls the memory controlling circuit 22, theimage display memory 24, the memory 30 and the like.

As described above, according to an embodiment of the present invention,a detailed comparison between images having a similar composition can beeasily made on site after the images are shot with a digital camera.Further, when the enlarging and moving operations are necessary withrespect to two (plural) images, a single operation applies to the exactarea in each image simultaneously, thereby improving the convenience ofuse. Still further, one operation applied to each image is applicablenot only in the play mode for normal image comparison but also in theREC view mode right after shooting the images, thereby allowingefficient use of recording media and reducing labor-hours in handlingthe images afterward.

Embodiment 2

Next, a Multi Function Printer (hereinafter referred to as an MFP)according to an embodiment of the present invention is described. FIG. 8is a drawing schematically showing an exemplary configuration of theMFP. The MFP includes an image input unit 300, a classifying unit 301, asimilar image searching unit 302, a correction value calculating unit303, a storage unit 304, a first display controlling unit 305, a seconddisplay controlling unit 306, a display unit 307, an operations unit308, and an output unit 309. The image input unit 300 inputs an imagevia, for example, a flat bed scanner and a network. The storage unit 304stores the input images. The display unit 307 lists the images stored inthe storage unit 304 and selects images to be output. The output unit309 reads and outputs the images stored in the storage unit 304.

In this embodiment, it is assumed that the storage unit 304 has a folderstructure as shown in FIG. 9 to classify images to be stored.

In this embodiment, it is assumed that an image is input by a flat bedscanner. A user designates a resolution and starts scanning an imagethrough the operations unit 308. Then the gray-scale information of theinput image is converted into an analog signal. The analog signal isfurther converted into a 8-bit (256-value) digital signal via a A/Dconverter. In the following, a term “an image” refers to a digitizedgray-scale signal.

When an image is input, the classifying unit 301 determines whether theinput image is a document image. When it is determined that the inputimage is not a document image, the input image is stored in the “other”folder shown in FIG. 9. The classification process is performed bycompressing an image without changing the aspect ratio of the image sothat the compressed image fits inside a 600×600 pixel rectangle, andinputting the compressed image into a Support Vector Machine (SVM, seeNon Patent Document 1) that has learned a value of each pixel.

The SMV is a classifier outputting “1” when the inner product betweenthe input vector (600×600 dimensional vector, in this embodiment) and aweight vector “a)” exceeds a prescribed threshold value and otherwise,outputting “−1”. The learning in SVM is a process of determining theweight vector “ω” and a threshold value “h”.

$\begin{matrix}{{sign}\left\lbrack {{\omega \; x} - h} \right\rbrack} & (3) \\{{{\sum\limits_{i = 1}^{N}\; {\alpha_{i}t_{i}}} = 0}{{0 \leq \alpha_{i} \leq \gamma},{i = 1},\ldots \mspace{11mu},N}} & (4) \\{{L_{D}(\alpha)} = {{\sum\limits_{i = 1}^{N}\; \alpha_{i}} - {\frac{1}{2}{\sum\limits_{i,{j = 1}}^{N}\; {\alpha_{i}\alpha_{j}y_{i}y_{j}{K\left( {x_{i},x_{j}} \right)}}}}}} & (5) \\{\omega = {\sum\limits_{i \in S}\; {\alpha_{i}t_{i}x_{i}}}} & (6) \\{h = {{\omega \; x_{0}} - t_{0}}} & (7) \\{{K\left( {x_{i},x_{j}} \right)} = {\exp \left( {{- C}{{x_{i} - x_{j}}}^{2}} \right)}} & (8)\end{matrix}$

Though detailed explanation of the learning in SVM is provided in NonPatent Document 1, the outline of the learning of the SVM is describedbelow. FIG. 14 is a drawing schematically showing the operation of theSVM. First, it is assumed that there are two vector groups (“◯” and “X”)as shown on left-hand side of FIG. 14. The learning in SVM may refer toan algorithm for determining a super plane (see right-hand side of FIG.14) appropriately dividing those two classes. In the learning algorithmin SVM, the term “appropriately dividing two vector groups” isequivalent to a maximize response capability or generalizationcapability when an unknown vector is input. To realize the capability, aSupport Vector in a boundary position of the two vector group is found,and the super plane is set so that a distance between the Super Vectorand the super plane. In an actual operation, however, there may be wrongteacher data, so that it is necessary to determine a parameter for errortolerance amount (soft margin). The above description is for linear SVM,and the actual teacher vector is not limited to a vector group that canbe linearly classified. However, a nonlinear case can be dealt with byprojecting a feature vector in higher order space (kernel trick) andsearching for a super plane in the space.

As a result, to realize the above, a Lagrangian multiplier vector αi isused that maximizes formula 5 using teacher data (xi, yi) under theconditions of formula 4. Then, the parameters “ω” and “h” are obtainedby using teacher data group “S” (this becomes a Support Vector)corresponding to an element other than “0” among the elements of theLagrangian multiplier vector αi and one arbitrary teacher data element(x0, y0) of the teacher vector group “S” (see formulas 6 and 7).

It should be noted that in the above formula, an expression “K(x,y)”refers to a Kernel function for realizing the Kernel trick. Though thereare provided various kernel functions, a Radial Basis Function (RBF) isused in this embodiment. The RBF is a function as shown in formula 8,where a symbol “C” refers to an arbitrary figure.

As described above, to perform the learning in SVM, it is necessary toset a parameter “γ” for an acceptable amount of the soft margin and avalue “C” for determining the RBF which is a Kernel function. In thisexample, values of γ=1, C=1000 are used.

Further, in this embodiment, an image is compressed and used as an inputvector. However, a feature amount may be calculated from the image andthe calculated feature amount may be used as the input vector. When thedimensions of the vector are reduced by calculating the feature amountof the vector, faster operation of the SVM can be achieved and thenumber of the teacher data sets necessary for the learning can bereduced.

Referring back to FIG. 8, the similar image searching unit 302determines a similarity level between an image in a document folder ofthe storage unit 304 and an input image. When there is an image having ahigh similarity level with the input image, the input image is stored inthe same folder as the image having the high similarity level. On theother hand, when there is no image having a high similarity level withthe input image, a new folder is created and the input image is storedin the created folder. It should be noted that the processes after thesimilar image searching unit 302 are preformed only when it isdetermined that the input image is a document image. By doing this, thefrequency of performing the similar image search is reduced and thespeed of processing an input image is increased. Further, it is assumedthat only one image from each folder under the document folder (see FIG.9) is used as an image in the storage unit 304 to be compared by thesimilar image searching unit 302. This is because it is alreadydetermined that images in the same folder have high similarity level toeach other. Therefore, it is not significant to compare the input imagewith plural images in the same folder. By reducing the number of imagesto be compared with an input image, the speed of processing an inputimage is increased.

As shown in FIG. 13, the similar image searching unit 302 includes abinarizing unit 401, a feature amount calculating unit 402, and acomparing unit 403. The binarizing unit 401 binarizes images to becompared with each other. The feature amount calculating unit 402calculates a feature amount called higher-order local autocorrelation asdescribed in Non Patent Document 2 for each of the images. The comparingunit 403 determines whether an Euclidean distance between the calculatedfeature amounts of the images exceeds a prescribed threshold value. Inthis embodiment the threshold value is set at 20.

Detailed explanation of the higher-order local autocorrelationextraction method is described in Non Patent Document 2. Therefore, anoutline of the method is described below. For simplicity purposes, aone-dimensional signal is considered. Then, an Nth autocorrelationfunction is defined as formula 9 with respect to N number ofdisplacements (a1, a2, . . . , aN) when a luminance value of an image atan reference point “r” is given as “I(r)”

x(a ₁,̂,a _(n))=∫I(r)I(r+a ₁)M(r+a _(N))dr  (9)

In the higher-order location autocorrelation, a signal of theautocorrelation function is extended to two dimensions. In this case,the number of orders “N” is limited to 2, and a range of thedisplacements is also limited to a local 3×3 pixel area near thereference point “r”.

To explain more specifically, a case of a pattern in No. 2 of FIG. 12 isdescribed as an example. A center pixel of the pattern is a noticedpixel. A multiplication is made between the noticed pixel and aright-hand black pixel in this case as a pixel related to the noticedpixel. Noticed pixels are scanned across the entire image and eachmultiplication result is accumulated to calculate the feature amount ofNo. 2. As described above, to calculate the higher-order localautocorrelation, up to two orders of correlation of pixels in a 3×3pixel area is considered. Therefore, when equivalent features due to byparallel displacement are removed, the number of all patterns becomes 25as shown in FIG. 12. Since each feature amount as described above of the25 patterns is calculated, the feature amounts become a 25-dimensionalvector. Further, since the obtained value changes according to thenumber of pixels in an image, the obtained feature amounts arenormalized by dividing by all pixels included in an image. It is knownthat the feature amount obtained as described above is a kind offrequency information of the image and the feature amount value isunchanged when the position of the object image is moved but thecontents are unchanged between images.

The correction value calculating unit 303 performs its operation whenthere is a similar image, and calculates a correction value so that thepositions of the searched for and found images correspond to those ofthe input image. This calculation is performed with a method similar tothat performed by the correction value calculating unit 503 described inthe first embodiment. However, in the first embodiment, an imageincluding in an area “R0” enlarged from the first image by a user issearched for in the second image. On the other hand, in this secondembodiment, a position (cx, cy) having the highest correlation value iscalculated by scanning the entire first image. In this case, asdescribed above, the first image may be directly scanned. However, whenthe first and the second images are compressed to “1/n” size, theposition (cx, cy) having the highest correlation value is calculated;then (cx, cy) is multiplied by “n” to obtain (n×cx, n×cy), so that theprocess can be performed faster. Further, the size of an input imagevaries according to the resolution set by a user. Therefore, before thecorrection value calculating unit 303 performs its process, an inputimage is enlarged or compressed so as to have a prescribed resolutionvalue. In this example of this embodiment, it is assumed that all imagesare enlarged or compressed so as to be a 300 dpi images. Further, anenlarge correction rate “c” defined as “300/a” where reference numeral“a” is a resolution value of an input image is calculated. Thecalculated (c′x, c′y) and “c” are stored in the storage unit 304 withthe corresponding image as a data pair. As described above, when animage is scanned, it is necessary to designate a resolution value.However, the resolution value may differ depending on the preference ofusers. However, by calculating the enlarge correction rate “c” as wellas calculating and storing the correction value of the position, a userdoesn't need to select a specific resolution value when scanning animage, thereby improving the convenience of use.

The image stored in the storage unit 304 can be displayed as a list bythe display unit 307. A user can change the images to be displayed as alist and select an output image through the operations unit 308. In thisembodiment, a touch panel is assumed to be used as the operations unit308. On the display unit 307, an enlarge/compression rate and paper sizeand the like are normally displayed. However, when a specific button ispressed, an image as shown in FIG. 9 is displayed. In the display, when,for example, the “ID001” folder is selected through the touch panel, animage in the “ID001” folder is transmitted to the first displaycontrolling unit 305 and the second display controlling unit 306. In thefirst display controlling unit 305, a specific area of the image is cutoff and transmitted to the display unit 307. On the other hand, in thesecond display controlling unit 306, the entire image is compressed andthe compressed image is transmitted to the display unit 307. Aboveprocesses are performed on all the images in the folder. It should benoted that prescribed values may be used for the enlarged position andthe enlarge rate. However, a folder and an enlarged area used by aprevious user may be used. It is assumed that when reference values ofthe cut off area are given as (xl, yb, xr, yt), the area cut off fromeach image with respect to the “c” and “(cx, cy)” corresponding to theeach image is given as (xl+cx, yb+cy, xr+cx, yt+cy).

As a result, a display as shown in FIG. 10 is displayed on the displayunit 307. Compressed images (processed by the second display image 306)and partially enlarged areas (images processed by the first displaycontrolling unit 305) with respect to six images are displayed. In thedisplay, when a slide bar on the right side is used, more than siximages can be easily seen by sliding the images upward and downward. Formore detailed description, FIG. 11 shows an enlarged image of the leftupper side of the display in FIG. 10. FIG. 11 shows a compressed imageof the entire input image on the left-hand side and a partialenlargement of the image on the right-hand side (side by side). Further,a rectangle shown in the left-hand-side image indicates an area to beenlarged in the right-hand-side image. This arrangement allows a user toeasily recognize which area is enlarged and displayed, thereby improvingthe convenience of use.

A user can enlarge or shrink the enlarged area by dragging the rightbottom of the enlarged area on the touch panel. Further, a user can movethe position of the enlarged area by dragging any position on theenlarged area other than the right bottom position. These operations aretransmitted to the first display controlling unit 305. In this case,each image in the folder is transmitted from the storage unit 304 to thefirst display controlling unit 305. The first display controlling unit305 cuts off the area to be enlarged designated by the user andtransmits the cut-off image to the display unit 307. By doing this,enlarged images with respect to all the displayed images are displayedbased on the area to be enlarged designated by the user. As describedabove, since not only is the enlarged area explicitly indicated on thecompressed image but also the area to be enlarged can be enlarged andshrunk by directly manipulating the rectangular area on the display, auser can intuitively enlarge and shrink and move the position of thearea to be enlarged, thereby improving the convenience of use.

As described above, when a user performs the same operation performed onone image with respect to an area to be enlarged in the other images aswell, the noticed area in all the other images can be accuratelyenlarged by correcting the differences of position and the enlarged rateof the area to be enlarged. Without performing such corrections, theuser has to make a fine adjustment on each image or scan each imageafter an accurate positioning process and set a predetermined resolutionwhen scanning images.

It should be noted that all or a part of the processes preformed by theclassifying unit 301, the similar image searching unit 302, and thecorrection value calculating unit 303 are not necessarily performedright after an image is input. For example, an image may be read fromthe storage unit 304 and be processed while no other jobs are running.The processes described above may require a long time depending on howthe processes are implemented. In such as case, a user has to wait forscanning one image to another. However, when all or a part of the aboveprocess is arranged to be performed while no other jobs are running onan MFP, or, for example, during other than business hours, the userdoesn't have to wait for completing the processes, thereby improving theconvenience of use.

Still further, in this embodiment, the entire image is shrunk by usingthe second display controlling unit 306 and the shrunken image isdisplayed on the display unit 307. However, when, for example, an imageformat capable of storing low-resolution images such as JPEG 2000 isused, it is not necessary to perform a separate shrinking process.

Still further, each of the above embodiments is realized by providing astorage medium (or recording medium) storing program code of software,capable of realizing functions according to an embodiment of the presentinvention as described above, in a system or an apparatus, and readingand executing the program code stored in the storage medium by acomputer (or a CPU and an MPU) of the system or the apparatus. In thiscase, the program code itself read from the storage medium and executedrealizes the functions of the embodiment of the present invention, andthe storage medium storing the program code constitutes an embodiment ofthe present invention. Further, when a computer executes the loadedprogram code, the described embodiment of the present invention isperformed, and based on the instructions from the program code, anoperating system (OS) running on the computer performs all or a part ofthe actual processes to realize the above described functions of anembodiment of the present invention. This case is also included in anembodiment of the present invention.

Still further, an embodiment of the present invention includes a casewhere program code loaded from a storage medium is written in a functionextension card of a computer or a memory as a function extension unitconnected to the computer, and a CPU included in the function extensioncard or the function extension unit performs all or a part of the actualprocesses, thereby realizing the functions of an embodiment of thepresent invention.

When an embodiment of the present invention is applied to the storagemedium, program code corresponding to the steps described above isstored in the storage medium.

The present invention is not limited to the above embodiments, andvariations and modifications may be made without departing from thescope of the present invention.

The present application is based on and claims the benefit of priorityof Japanese Patent Application No. 2007-124467, filed on May 9, 2007,the entire contents of which are hereby incorporated herein byreference.

1. An image processing apparatus comprising: an image input unitinputting an image; a recording unit recording the image input by theinput image unit; a display unit displaying the image recorded in therecording unit; a first display controlling unit simultaneouslyperforming a process selected from the group including enlarging animage and moving an image with respect to specific plural of the imagesdisplayed on the display unit; and a correction value calculating unitcalculating a correction value with respect to the process in the firstdisplay controlling unit in accordance with the selected process;wherein the process in the first display controlling unit is correctedin accordance with the correction value.
 2. The image processingapparatus according to claim 1, wherein the correction value calculatedby the correction value calculating unit is first recorded in therecording unit and read when a corresponding image is processed in thefirst display controlling unit.
 3. The image processing apparatusaccording to claim 1, further comprising: an operations unittransmitting an instruction from a user to the apparatus, wherein theoperations unit reads an image from the recording unit in accordancewith a user's operation, and the operations unit controls the firstdisplay control unit in accordance with a user's operation.
 4. The imageprocessing apparatus according to claim 1, further comprising: animaging unit imaging an image, wherein the first display controllingunit has a function to perform a REC view operation displaying the imageimaged by the imaging unit right after an imaging operation by theimaging unit, and when receiving an instruction to continue the REC viewoperation, the specific plural images are automatically displayed on thedisplay unit.
 5. The image processing apparatus according to claim 1,wherein the plural images are any of plural images sequentially shot ina bracket shooting, plural images shot in a sequential shooting, andplural images shot within a prescribed period of time.
 6. The imageprocessing apparatus according to claim 1, further comprising: an imagesimilarity level calculating unit calculating a similarity level of animage recorded in the recording unit, wherein the correction value iscalculated using an image determined to have a high similarity levelwith at least the input image.
 7. The image processing apparatusaccording to claim 1, wherein the correction value calculating unitincludes an enlargement ratio correction value calculating unit.
 8. Theimage processing apparatus according to claim 1, further comprising: asecond display controlling unit processing an image, wherein all or apart of the images read from the recording unit are transmitted to thefirst display controlling unit and the second display controlling unit,and an image processed by the first display controlling unit and anotherimage processed by the second display controlling unit aresimultaneously displayed on the display unit.
 9. The image processingapparatus according to claim 8, wherein the second display controllingunit outputs an image having lower resolution than the image input tothe second display controlling unit.
 10. The image processing apparatusaccording to claim 8, wherein, a correspondence between the two imagesprocessed by the first display controlling unit and the second displaycontrolling unit, respectively, is clearly displayed in a rectangle onat least one of the images.
 11. The image processing apparatus accordingto claim 10, wherein an operation performed by the first displaycontrolling unit is changed and contents displayed on the display unitare updated by an operation of pointing and dragging the rectangledisplaying the correspondence between the two images.
 12. The imageprocessing apparatus according to claim 1, further comprising: aclassifying unit determining a type of the image.
 13. The imageprocessing apparatus according to claim 12, wherein following processesare switched depending on whether a type of the image is a prescribedtype determined by the classifying unit.
 14. The image processingapparatus according to claim 12, wherein the classifying unit determinesa type of the image before determining a similarity level.
 15. An imageprocessing method comprising: an image input step of inputting an image;a recording step of recording the image input by the input image unit; adisplay step of displaying the image recorded in the recording unit; afirst display controlling step of simultaneously performing a processselected from the group including enlarging an image and moving an imagewith respect to specific plural of the images displayed on the displayunit; and a correction value calculating step of calculating acorrection value with respect to the process in the first displaycontrolling unit in accordance with the selected process; wherein theprocess in the first display controlling unit is corrected in accordancewith the correction value.
 16. The image processing method according toclaim 15, wherein the correction value calculated by the correctionvalue calculating unit is first recorded in the recording unit and readwhen a corresponding image is processed in the first display controllingunit.
 17. A program causing a computer to perform the image processingmethod according to claim
 15. 18. A computer-readable recording mediumstoring a program causing a computer to perform the image processingmethod according to claim 15.